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ABSTRACT 


This  report  addresses  cost  growth  problems  that  have  been  experienced  with  CPIF 
contracts  in  the  Army  Materiel  Command,  especially  those  problems  related  to  the  incentive 
structure  itself.  The  data  base  is  composed  of  53  randomly  selected  CPIF  contracts 
completed  between  1964  and  1971  with  initial  price  of  $500,000  or  more. 

The  important  major  findings  are  tint: 

a.  The  use  of  the  most  probable  cost  for  target  costs  (directed  by  ASPR)  as  opposed 
to  expected  cost,  produces  about  20  percent  contract  cost  growth. 

b.  There  is  a positive  correlation  between  contractor’s  share  of  underrun  and 
contractual  adjustments  and  a negative  correlation  between  overrun  and  the  contractor’s 
share  for  overruns. 

c.  The  contractor's  share  of  underrun  and  overrun  is  less  than  the  negotiated  share, 
on  the  average,  while  his  profit  for  final  costs  which  are  above  the  upper  limit  of  the  range 
of  incentive  effectiveness  is  occasionally  far  greater  thi  n the  negotiated  minimum  profit. 


SUMMARY 


1.  Background  and  Purposes. 

In  previous  cost  growth  research  performed  in  this  office,  the  CPIF  type  of  contract 
was  found  to  be  associated  more  frequently  end  in  greater  intensity  with  contract  cost 
growth  than  any  other  contract  type.  Furth-i  analysis  of  this  research  finding  suggested 
some  potentially  useful  insights  regarding  relationships  between  cost  growth  on  the  one 
hand  and  share  ratios,  target  costs,  overruns/underruns,  and  the  ranges  of  incenti'e 
effectiveness  on  the  other  hand.  The  purpose  of  this  study  is  to  provide  a closer  exeminatio  \ 
of  these  relationships. 

2.  Nature  and  Scope  of  Study. 

The  study  is  analytic  in  nature  and  many  loupccs  have  been  utilized  in  generating  data 
including  forms  DD  1500,  DD  350,  Contractor  Performance  Evaluation  Reports  (CPE),  and 
the  contract  file  data  extracted  from  a random  sample  of  300  AMC  procurements.  These 
data  were  used  to  construct  the  cost  growth  profile  of  contract  types,  type  of  work, 
commodities,  and  many  other  categories. 

3.  Findings. 

The  findings  of  this  report  are  listed  below. 

a.  The  negotiated  share  ratio  and  the  estimated  share  ratio  (aj  estimated  from  the 
final  cost  and  profit)  are  not  correlated.  This  information  is  important  to  anyone 
conducting  studies  involving  CPIF  contracts. 

b.  The  use  of  “most  probable  costs”  (directed  by  ASPR)  as  opposed  to  the  use  of 
expected  costs  in  negotiation  procedures  induces  about  20  percent  contract  cost  growth. 

c.  There  i.'  . positive  correlation  between  the  dollar  cost  of  conn-actual  adjustments 
and  the  contractor’s  share  of  the  underrun,  and  there  is  a negative  correlation  between  the 
overrun  and  the  contractor’s  share  for  overrun.  This  indicates  that  the  slope  of  the  “share 
line”  affects  overruns  and  underruns  in  the  intended  way.  It  was  also  found  that  the  cost  of 
contractual  adjustments  as  a percent  of  the  contract’s  initial  cost  is  highly  correlated  witli 
the  difference  hetween  the  contractor’s  share  of  overrun.  These  collective  results  constitute 
a partial  confirmation  of  the  existence  of  the  conu actor  buy-in. 

d.  The  share  of  both  the  underrun  and  the  overrun  which  the  contractor  actually 
receives  is,  on  the  average,  less  than  the  negotiated  share  within  and  above  the  range  of 
incentive  effectiveness.  In  other  words,  the  contractor  is  receiving  more  profit  for  overruns 
and  less  profit  for  underruns  than  that  indicated  by  the  share  line. 

4.  Recommendations. 

The  results  of  this  study  are  cf  such  a nature,  that  clear-cut  policy  recommendations 
would  he  exceedingly  difficult  to  make.  Therefore,  the  recommendations  arc  pointed  at 
either  senior  Army  policy  makers  for  consideration  :n  policy  formulation,  or  else  for 
analysts  and  researchers  who  offer  advice  to  the  policy  makers  or  who  conduct 
investigations  in  logistics  problems. 


VII 


a.  It  is  recommended  that  senior  DA  procurement  analysts  be  made  aware  of  the 
relationship  between  the  mean  and  the  mode  in  CPIF  cost  data.  The  obvious  but  simplistic 
recommendation  that  “expected”  costs  should  lie  used  instead  of  “most  probable”  costs 
coll  be  avoided  here,  because  of  the  possible  effect  of  the  resulting  higher  target  costs  on  the 
final  costs.  Any  further  policy  recommendation  will  require  more  study  temper*  d with 
sound  procurement  judgment 

b.  It  is  recommended  that  procurement  analysts  be  made  aware  of  the  disparity 
between  the  contractors  negotiated  share  of  overrun  and  underrun  and  the  (smaller)  share 
which  lie  receives  on  the  average.  A more  extensive  recommendation  will  not  be  given  since 
this  disparity  works  to  the  advantage  of  the  Government  in  the  underrun  situation.  There  is 
also  the  possibility  of  the  existence  of  hidden  “trade-offs”  between  the  Government  and  the 
contractor,  which  are  not  measurable  from  the  data,  but  which  work  to  the  benefit  of  the 
Government. 

c.  It  is  recommended  that  studies  performed  within  the  Department  ot  the  Army 
involving  the  affect  of  the  share  ratio  in  CPIF  contracts  on  cost  growth  should  use  the 
negotiated  share  ratio  obtained  from  the  contract  files.  Analysis  which  is  based  on  share 
ratios  estimated  from  the  final  cost  data  may  be  highly  inaccurate. 


CHAPTER  I 


INTRODUCTION 


A.  Purpose. 

The  general  purpose  of  this  study  is  a close  examination  of  several  problem  areas  and 
relationships  related  to  CPIF  contracts.  In  particular,  this  study  is  aimed  at  investigating  the 
relationship  between  cost  growth  patterns  and  (a)  the  magnitude  of  the  share  ratio;  (b) 
target  costs  derived  from  estimated  most  probable  costs,  as  opposed  to  estimated  expected 
costs;  (c)  the  difference  in  the  share  ratio  between  underrun  and  overrun;  and  (d)  the  range 
of  incentive  effectiveness  along  with  the  contractor's  management  of  contract  modifications. 

Potentially  useful  insights  regarding  these  relationships  were  observed  in  previous  cost 
growth  research,  [2,  4,  7,  8,  10, 15].  For  example,  the  following  statement  was  made  in  the 
immediately  preceding  cost  growth  report  1 8 ] : 

“Of  the  net  6.9  percent  cost  growth  attributable  to  cost 
overrun,  22  overruns  increased  cost  growth  by  9.0  percent, 
while  the  28  underruns  decreased  cost  growth  by  only  2.1 
percent.  Overruns  and  underruns  were  fairly  evenly 
distributed  within  the  cost  range  in  which  cost  incentive 
features  of  the  contract  were  effective.  Once  the  cost 
exceeded  the  upper  range  of  incentive  effectiveness  however, 
large  overruns  were  recorded.  Moreover,  no  instances  were 
observed  where  the  actual  costs  fell  below  that  level  where 
the  contractor  earned  maximum  allowable  fee.  Thus,  it 
appears  that  contractors  made  no  attempt  to  control  costs 
when  the  cost  was  significantly  outside  the  incentive  range. 

Additional  analysis  is  being  performed  in  this  area." 

This  statement  prompted  the  study  of  (d)  above. 

B.  Scope  and  Method. 

The  study  is  primarily  a statistical  analysis  of  5.1  randomly  selected  CPIF  contracts 
performed  for  the  Department  of  the  Army  between  1964  and  1971.  All  contracts  in  the 
sample  were  definitized  at  $500,000  or  more.  The  procurements  studied  include  contracts 
with  contractors  share  ratios  ranging  from  9.86  percent  to  50  percent  The  relationships 
between  cost  estimates,  final  cost,  share  ratios,  and  ranges  of  incentive  effectiveness  are 
studied  by  means  of  correlation  and  regression  analysis,  “piecewise  linear  regression,” 
analysis  of  variance  and  the  estimation  of  density  functions  and  modes.  The  appropriateness 
of  each  of  these  quantitative  methods  to  the  particular  relationship  being  studied  is 
explained  along  with  the  analysis  in  the  text.  Of  special  note,  the  “piecewise  linear 
regression’’  technique  was  adopted  especially  for  this  study  to  examine  the  effect  of  the 
range  of  incentive  effectiveness  because  the  share  line  is  piecewise  linear. 


C.  Desired  Objectives. 

The  desired  objectives  of  the  study  at  the  outset  were  to  determint. 

1.  Whether  or  not  there  is  a tendency  for  contractors  to  attempt  to  control  costs 
only  so  long  as  the  cost  is  within  the  range  of  incentive  effectiveness. 

2.  That  if  there  is  such  a tendency  (implying  that  the  contractor  has  a great  deal  of 
control  over  the  final  cost),  is  it  possible  to  detect  this  control,  perhaps  in  the  form  of  a 
buy-in? 

3.  What  ultimate  effect  the  practice  of  using  most  probable  costs  for  target  costs  has 
on  cost  growth. 

4.  Whether  or  not  the  magnitude  of  the  contractor’s  share  is  related  to  cost  growth. 
If  no  relationslup  is  detected,  is  this  a result  of  both  the  share  and  cost  growth  l>eing 
influenced  by  the  uncertainty  in  the  contract? 

5.  Whether  or  not  the  degree  of  break  in  the  share  between  overrun  and  underrun  is 
related  to  cost  growth.  If  no  relationship  is  detected,  is  this  a result  of  both  the  break  in  the 
share  line  and  cost  growth  being  influenced  by  the  uncertainty  in  the  contract? 

6.  Whether  the  range  of  incentive  effectiveness  should  be  extended  or  not. 

7.  Whether  the  correlation  between  the  negotiated  share  ratio  and  the  share  ratio  as 
“estimated”  from  the  contract  final  cost  and  profit  figures  is  high  enough  to  warrant  using 
this  estimated  share  ratio  in  analyses  which  arc  based  on  negotiated  share  ratios. 

3.  Description  of  Incentive  Structure. 

The  structure  of  the  incentive  share  ratio  (S/R)  and  the  range  of  incentive  effectiveness 
(RIF.)  for  cost  and  multiple  incentive  contracts  is  discussed  thoroughly  in  the  “Incentive 
Contracting  Guide”  [4],  Briefly,  the  share  ratio  reflects  the  percent  of  the  difference 
between  target  cost  and  final  cost  which  is  given  to  the  contractor  if  an  underrun  is 
experienced,  or  taken  from  the  contractor  in  the  case  of  an  overrun.  In  order  to  make  this 
system  reasonable,  a minimum  profit  and  a maximum  profit  are  imposed  and  the  costs 
which  correspond  to  maximum  and  minimum  profit  are  referred  to  as  the  lower  and  upper 
limits  of  the  RIE,  respectively.  The  RIE  is,  therefore,  that  range  of  values  of  the  cost  over 
which  the  S/R  operates.  The  share  ratio  is  expressed  as  a ratio  of  the  form  X/(100-\),  where 
y.  is  the  percent  of  the  overrun  (underrun)  which  the  Government  pays  (keeps).  Rv  law,  the 
maximum  profit  that  may  be  earned  in  a CPIF  contract  is  15  percent  of  the  target  cost  for 
R&I)  contracts  and  10  percent  of  the  target  cost  for  others  (ASPP  3-405.6).  Thus,  the  limits 
of  the  RIE  arc  usually  specified  in  the  contract  by  the  maximum  and  minimum  profit, 
expressed  as  a percent  of  target  cost. 

It  frequently  happens  that  a “broken”  S/R  is  negotiated  for  a particular  CPIF  contract. 
That  is,  the  share  ratio  for  an  overrun  (O/R)  may  be  different  from  the  S/R  for  an  undenun 
(U/R).  In  fact,  it  also  happens,  that  the  S/R  may  vary  for  different  values  of  the  overrun  (or 
U/R).  For  examples,  the  S/R  could  be  90/10  if  the  O/R  is  up  to  $1,000,000;  R5/15  if  the 
O/R  is  between  $1M  and  $2M  and  80/20  if  O/R  is  over  $2M,  hut  less  than  the  maximum 
limit  prescribed  by  the  termg  of  the  contract. 
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The  exact  value(s)  of  the  S/R  is  usually  determined  by  negotiation,  and  theoretically 
reflects  the  amount  of  risk  inherent  in  the  work  to  be  performed  under  the  terms  of  the 
contract.  A contractor  who  believes  that  there  is  a grea.  deal  of  risk  or  uncertainty  involved 
in  a particular  contract  will,  on  the  average,  not  accept  an  S/R  as  “largo”  as  the  S/R  that  he 
would  accept  on  a contract  in  which  'here  is  very  little  risk.  In  other  words,  a contractor 
may  be  willing  to  accept  a 30  percer  t or  even  a 50  percent  share  of  O/R  for  a contract  to 
which  he  attaches  little  or  no  uncertainty  but  would  accept  no  “more”  than  a 10  percent 
for  a very  risky  contract. 


CHAPTER  II 

CORRELATION  OF  NEGOTIATED  SHARE  RATIO  WITH  THE  COMPUTED  SHARE  RATIO 

The  research  contained  in  this  study  was  condu  ed  to  -nswer  several  specific  questions 
about  the  cost  growth  profile  of  the  CPIF  coat,  act  type.  (See  Chapter  I.)  These  questions 
involve  the  negotiated  share  ratio  and  range  of  incentive  effectiveness  either  directly  or 
indirectly.  Several  studies  have  been  written  which  treat  these  or  similar  questions,  but  they 
all  appear  to  use  estimates  of  the  share  ratio  instead  of  obtaining  the  actual  negotiated  share 
ratio.  Admittedly,  obtaining  the  negotiated  figures  is  far  more  costly  and  time  consuming 
than  estimating  them.  There  are,  however,  several  reasons  why  the  estimated  share  ratio 
could  very  reasonably  be  expected  to  differ  from  the  negotiated  share  ratio.  One  reason  is 
that  the  range  of  incentive  effectiveness  produces  a break  point  in  the  “share  line.” 

Most  studies  estimate  the  share  ratio  from  the  final  contract  cost  and  profit  and  then 
treat  it  in  the  analysis  as  if  it  were  the  negotiated  share  ratio  which  presumably  affects  the 
final  cost  and  profit  (and,  therefore,  affects  the  estimated  share  ratio.) 

If  the  estimated  share  ratio  is  always  in  very  close  agreement  with  the  negotiated  share 
ratio  (say  within  1 percent  or  2 percent)  then  substituting  one  for  the  other  would  seem  to 
be  an  acceptable  procedure.  If  the  agreement  is  not  close,  then  the  practice  of  utilizing  these 
estimates  in  an  analysis  is  open  to  serious  question.  It  was  decided,  therefore,  to  begin  the 
study  with  a correlation  analysis  between  the  estimated  and  negotiated  share  ratios.  The 
results  of  this  analysis  would  determine  the  need  and  appropriateness  of  examining 
additional  relationships,  such  as  those  cited  in  the  purpose  stated  above. 

The  incentive  share  ratio  is  related  to  the  contract  costs  and  profit  in  the  following 

way. 


Let,  1 1 A = Adjusted  target  profit 

77  p = Final  profit 

CA  = Adjusted  target  cost 

Cp  = Final  cost 

S = Incentive  share  ratio 

Then,  77 \ - TT  F = s(r‘F^\)» and  80  > ' 

s = - n £ (1) 

Cp  • CA 

Since  the  profit  awarded  on  CPIF  contracts  is  rarely  precisely  the  same  as  that  given  b)  the 
negotiated  share  line,  formula  (1)  will  not  in  general  agree  with  the  negotiated  share  ratio. 
The  most  likely  causes  of  this  disparity  are  multiple  incentives,  broken  share  lines,  and  the 
final  contract  renegotiation  and  closeout.  These  are,  however,  not  the  only  causes. 


E 
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Figure  1 is  * scatter  plot  of  the  computed  share  ratio  (from  formula  (1))  versus  the 
negotiated  share  ratio.  Since  the  negotiated  share  ratio  for  overruns  will  frequently  differ 
from  the  underrun  negotiated  share  ratio  on  a given  contract,  care  was  taken  to  use  the 
appropriate  figure  in  the  correlation  analysis.  (Overruns  and  underruns  were  analyzed 
separately  to  see  if  this  would  improve  the  resulting  estimates).  Furthermore,  in  order  to 
avoid  ambiguity,  a contract  which  experienced  an  overrun  and  which  had  a broken  share 
ratio  for  overruns,  was  not  included,  and  similarly  for  underruns. 

Table  I gives  the  results  of  the  analysis.  The  t- values  are  computed  from  the  formula: 


t = 


t 


C'c/'  ? 


y>-z 


where  r is  the  estimated  correlation  coefficient: 


Notice  that  the  t statistics  are  all  quite  low.  The  largest  one  fails  to  be  significant  at 
| even  the  .20  level.  This  indicates  that  the  correlation  between  the  negotiated  share  ratio  and 

2 the  share  ratio  as  computed  after  the  fact  from  formula  (1)  is  practically  zero. 

I The  conclusion  to  be  drawn  from  this  result  is  that  estimates  of  share  ratios  from  cost 

| data  (such  as  the  form  I)D  1500)  are  extremely  inaccurate  and  inferences  obtained  through 

; their  use  are  invalid.  This  conclusion  is  not  an  indictment  of  the  accuracy  of  the  DD  1500 

l data,  but  rather  points  out  the  fact  that  contract  cost  incentive  payments  are  affected  by 

l such  things  as  f*  hedule  and  performance  incentives,  broken  share  ratios,  and  the  range  of 

v incentive  effect!,  ness. 


IMPUTED  SHARE  RATIO  VS.  NEGOTIATED  SHARE  RATIO 
(Does  Not  Include  Broken  Share  Ratio) 
(Notice  the  Negative  Imputed  Share  Ratio) 


50  100  200 


Negotiated  Share  Ratio 


FIGLRE  1 


CHAPTER  m 


THE  ROLE  OF  COST  ESTIMATES  IN  THE  COST  GROWTH  OF  CPIF  CONTRACTS 


A.  Introduction. 

There  are  many  established  aa  well  as  potential  but  not  necessarily  positively  identified 
producers  of  cost  growth.  One  cause  which  is  often  mentioned  but  rarely,  if  ever,  carefully 
analyzed  for  its  effect  on  cost  growth  is  the  cost  estimate  itself.  The  reason  for  this  is  not 
lack  of  interest  but  is  due  to  the  lack  of  appropriate  data.  This  is  f result  of  the  fact  that  the 
notion  of  a “true’*  contract  cost  (in  the  absolute  sense)  is  so  nebulous. 

If  it  were  possible  for  several  contractors  to  work  simultaneously  and  independently  on 
identical  contracts,  with  no  changes  in  scope,  quantity,  engineering,  or  schedule  issued  by 
the  Government,  and  with  no  price  competition  among  the  contractors,  then  there  would 
probably  be  several  different  cost  estimates  for  the  various  contractors  and  several  different 
final  costs  for  each  of  the  contracts.  Furthermore,  the  cost  estimates  and  final  cost 
experienced  by  a given  contractor  would  almost  certainly  differ! 

It  seems  clear  from  the  preceding  paragraph,  tliat  a direct  comparison  of  a cost 
estimate  with  its  “true*'  co;.t  is  not  possible.  This  suggests  that  an  indirect  analysis  based  on 
statistical  methods  might  be  advantageous.  Therefore,  the  cost  estimate  and  the  negotiated 
cost  of  a contract  will  be  treated  as  if  they  were  random  variables.  (The  word  random  in  tliis 
context  does  not  mean  haphazard  but,  rather,  unpredictable.)  A random  variable  (r.v.)  is 
usually  characterized  by  descriptive  parameters  such  as  “expected  value”  (overall  average) 
and  “variance"  (spread  or  dispersion).  The  true  cost  mentioned  previously,  corresponds 
most  closely  with  the  concept  of  expected  value. 

The  “Armed  Services  Procurement  Regulation”  (ASPR)  and  the  “Incentive 
Contracting  Guide"  are  written  with  this  in  mind. 

The  following  three  quotations  are  taken  from  the  “Incentive  Contracting  Guide,” 
October  1969,  FM  38-34. 


(1)  The  ingredients  of  a cost-plus-incentive-fee  contract 
(CPIF)  are: 


(i) 

<«) 

(iii) 

(iv) 

(v) 


Target  Cost  (the  most  probable  cost  for  target  performance) 
Target  Fee  (a  reasonable  feo  for  target  performance) 
Maximum  Fee  (subject  to  Agency  control) 

Minimum  Fee  (may  be  a “negative  fee”) 

Share  Formula  (the  arrangement  for  establishing  final  fee) 
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You  will  Have  observed  that  the  definition  of  target  coat 
above  as  the  most  probable  coat  for  target  performance  is 
different  than  the  definition  given  previously  for  target  coat 
under  the  fixed-price  incentive  contract  coverage  as  the  cost 
“against  which  to  measure  final  cos  „v.”  For  either  contract 
type  the  latter  description  of  “most  probable  cost”  applies  to 
target  coat.  (P.  79—80.) 

(2)  Much  discussion  centers  on  the  question  “What  is  a 
‘good*  target?”  It  has  been  suggested  that,  “A  good  target  cost 
is  one  about  which  both  parties  can  agree  there  is  an  equal 
chance  of  either  overrunning  or  underrunning  basing  then- 
judgment  on  all  complete  and  current  facts  available  at  a 
point  in  time,  . . .the  estimated  target  cost  should  be  one  of 
equal  chance  of  overrunning  or  underrunning,  not  equal 
magnitude.  The  idea  of  symmetry  has  somehow  crept  in  and 
people  tend  to  say  a target  cost  is  good  + or  - 20  percent. 
This  is  rarely  true.  The  magnitude  of  the  potential  overrun 
usually  will  not  equal  the  magnitude  of  the  potential 
underrun.  (P.  85.) 

(3)  The  target  cost  should  represent  the  best,  mutually 
determined  estimate  of  what  costs  will  actually  be  when 
incurred,  or,  stated  another  way,  that  target  cost  should 
represent  that  figure  at  which  there  is  equal  probability  of 
cither  a cost  underrun  or  overrun.  (P.  87.) 


Several  important  observations  should  be  made  at  this  point.  Quotation  (1)  refers  to  a 
target  cost  as  “the  most  probable  cost."  The  most  probable  value  of  a random  variable  is 
called  its  MODE  [5J.  Quotation  (2)  says  that  a good  target  cost  is  one  for  which  Che 
probability  of  an  overrun  or  underrun  arc  equal.  This  value  of  a random  variable  is  its 
MEDIAN,  [5].  Finally,  quotation  (3)  says  that  the  target  cost  should  represent  the  best 
estimate  of  what  costs  will  actually  be.  This  ordinarily  means  the  expected  value  or  MEAN 
of  a random  variable.  (The  remainder  of  quotation  (3)  then  says  that  this  "expected  cost”  is 
the  “median”  cost  of  quotation  (2),  but  since  the  term,  “estimate  of  what  costs  will  actually 
he”  is  vague,  nothing  more  will  be  said  about  this  last  point.) 

In  general,  unless  the  distribution  of  a continuous  random  variable  is  symmetric,  its 
mean,  median,  and  mode  will  all  differ  from  one  another.  The  claim  is  made  in  quotation 
(2)  that  overruns  and  underruns  are  not  symmetric,  which  then  means  that  the  statements 
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made  in  the  quotations  are  contradictory.  Since  the  meaning  of  the  mean,  median,  and 
mode  are  of  little  interest  to  the  procurement  Analyst,  this  diacuaeion  at  first  aeema  hardly 
worthy  of  development  here.  Trere  is,  however,  one  very  important  conaequence  of  aQ  of 
thia  for  coat  growth  meawremeota. 

One  measure  of  the  asyrmetry  of  the  distribution  of  a r.v.  is  its  akewness ,7^ , which  is 
defined  to  be  the  difference  of  the  mean  p and  mode  divided  by  the  standard  deviation, 
(S.D.)  (3]. 

That  is, 

n-  mean  • mode 


If  the  mean  is  greater  than  the  mode,  then  the  skewness  is  positive.  One  type  of 
distribution  which  has  positive  skew  is  one  which  has  a relatively  long  tail  to  the  right  and  is 
bounded  on  the  left. 

Cost  growth  is  customarily  estimated  from  sample  averages  and  the  sample  average  is 
an  estimate  of  the  “mean"  cost  growth.  Since  the  means  of  both  cost  growth  and  overruns 
arc  greater  than  their  respective  modes  (Section  R),  then  the  use  of  most  probable  costs  as 
target  costs  will  invariably  result  in  cost  increases  as  computed  by  average  cost  growth. 

B.  The  Skewness  and  Mode  of  CPIF  Overruns  and  Adjustments. 

In  an  effort  to  estimate  just  how  much  cost  growth  is  generated  by  “modal"  estimation 
of  contract  costs,  it  was  decided  to  estimate  these  from  the  sample.  The  estimation  of  the 
;aode  of  a r.v.  is  relatively  new.  Parzen  [ 11]  has  obtained  such  a procedure.  (See  appendix  I 
for  a brief  description.) 

Figure  2 is  a sketch  of  the  estimate  of  the  distribution  of  cost  growth  as  a percent  of 
initial  cost  for  the  entire  sample  of  53  contracts.  The  mean  and  mode  are  approximately  57 
percent  and  37  percent  cost  growth  respectively.  The  overruns  distribution  and  mode  are 
given  in  figure  3 with  mean  and  mode  .04  and  .008  respectively. 

Since  eight  of  the  contracts  did  have  quantity  changes,  the  procedure  was  repeated  for 
cost  growth  less  the  quantity  and  for  the  contractual  adjustments  less  overrun  and  less 
quantity  changes.  In  both  cases,  the  mean  is  about  20  percent  greater  than  the  mode,  as 
depicted  in  figures  4 and  5. 

Since  quantity  changes  are  apt  to  produce  overruns,  it  would  be  useful  to  examine  the 
distribution  of  cost  growth  without  the  quantity  changes  and  without  that  part  of  the 
overrun  which  the  quantity  changes  produce.  This  data  is  not  available,  but  can  be  estimated 
if  one  is  willing  to  prorate  quantity  changes  so  that  the  ratio  of  overruns  produced  the 
quantity  changes  to  the  actual  (recorded)  ove-run  is  the  same  as  the  ratio  of  the  dollar  value 
of  the  quantity  changes  to  the  dollar  value  of  the  initial  target  cost  plus  quantity  changes. 
That  is  to  say,  if  a contract  experiences  a quantity  change  which  doubles  the  original 
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quantity  then  50  percent  of  the  overrun  is  due  to  the  quantity  change  and  the  remaining  50 
percent  ia  the  overrun  that  “would  have  been,”  without  any  quantity  change.  That  fraction 
of  the  overrun  which  ia  due  only  to  the  quantity  ordered  in  the  original  contrart  will  be 
referred  to  aa  overran  prorated  to  initial  coat 


Let: 


CI 

CA 

Q 

R 


Then, 


Initial  target  coat 
Adjurted  target  coat 
Coat  of  quantity  increase 
Overran 

Overrun  prorated  to  C] 


^ 2_ 

R Cj  + Q, 


The  overrun  and  quantity  changes  may  be  expressed  aa  a percent  of  initial  cost. 


= “Percent”  overrun 


= “Percent”  overrun  prorated 


= “Percent”  Quantity  changes. 


Finally, 

i i iWj 
c,  C,  * 1 + 0/C, 
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Cost  As  % of  Initial  Cost 


COST  GROWTH  DENSITY;  QUANTITY  CHANGES  REMOVED 


Quantity  As  % of  Initial 


Adjustments  Less  Quantity  As  % of  Initial  Cost 


In  a similar  way,  the  overruns  can  be  corrected  or  prorated  on  the  basis  of  the  adjusted 
cost,  LA,  or  on  the  basis  of  the  contract  adjustments,  - Cj.  The  percent  cost  growth  may 
then  be  corrected  by  subtracting  the  original  overrun  and  adding  the  prorated  overrun 
figure.  Since  only  eight  of  the  S3  CPIF  contracts  experienced  quantit  increases  and  of 
these,  four  experienced  no  contractual  adjustments  other  than  quantity  changes  while 
another  had  less  than  1 percent  quantity  increases,  only  the  overrun  prorated  to  initial  cost 
and  adjustments  were  explored.  In  all  cases,  the  overrun,  contractual  adjustments  and  cost 
growth  exhibited  a positive  skew. 

Figures  6,  7,  and  8 give  the  overruns  and  cost  growth  corrected  to  initial  cost  and 
adjustments  for  the  sample.  Notice  that  the  difference  between  mean  and  mode  of  cost 
growth  is  about  21  percent  in  both  cases.  Estimation  of  target  costs  with  most  probable 
costs  should  then  result  in  approximately  21  percent  cost  growth  on  the  average,  even  with 
no  changes  in  scope  or  work  definition.  The  remaining  26  percent  or  27  percent  cost  growth 
is  the  average  cost  growth  that  would  be  experienced  if  expected  costs  and  not  most 
probable  costs  were  used  ip  procurements,  and  if  the  final  cost  is  independent  of  the  target 
cost. 
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Overrun  Less  Quanti 
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Overrun  Less  Quantity  (Pro-Rated)  As  % of  Initial  Cost 
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C.  The  Cost  Estimation  Dilemma. 

From  a statistical  point  of  view,  the  quality  of  an  estimator  may  in  some  cases  be 
improved  by  “underestimation,”  if  quality  means  “closer  more  often."  (Appendix  II)  This  is 
frequently  the  case  when  the  distribution  has  one  mode  and  is  skewed  to  the  right  as 
contract  costs  are.  This  measure  of  quality  is  not  the  same  as  the  simple  comparison  of 
average  with  the  actual  value  which  is  to  be  estimated  (such  as  the  measurement  of  cost 
growth),  but  is  somewhat  more  complicated  (Appendix  II).  Stated  simply  this  means  that 
using  most  probable  costs  for  target  costs  is  a statistically  sound  estimation  procedure,  even 
though  it  results  in  underestimating  the  costs.  But  average  coet  growth  will  be  increased  due 
to  this  underestimation.  The  following  example  is  given  to  illustrate  this  point. 

Suppose  that  cost  analyst  A,  and  coet  analyst  B use  different  techniques  for  estimating 
costs.  Coet  analyst  A consistently  underestimates  costs  by  4 percent  to  6 percent,  and  on 
the  average  his  estimates  are  5 percent  too  low.  On  the  other  hand,  coet  analyst  B is  off  by 
as  much  as  25  percent  above  or  below  cost,  but  his  overall  average  coet  estimate  is  precisely 
accurate.  A contracting  officer  would  almost  certainly  prefer  the  services  of  cost  analyst  A, 
while  the  chief  budgeter  for  the  Department  of  the  Army  might  prefer  to  utilize  cost  analyst 
B in  preparing  the  budget  for  the  next  fiscal  year. 

There  is  one  other  benefit  to  modal  (under)  estimation.  Tliere  is  good  reason  to  believe 
that  more  realistic  coet  estimates  obtained  by  increasing  target  costs  will  not  reduce  cost 
growth,  [12, 13]. 

If  final  contract  prices  would  remain  the  same  regardless  of  the  initial  target  cost,  then 
overall  cost  growth  could  be  eliminated.  In  fact,  systematic  overestimation  would  result  in 
what  would  appear  to  be  tremendous  savings  in  tax  dollars.  Since  this  is  not  the  way  the 
procurement  game  is  played  [14],  then  maintaining  the  targets  at  lower  levels  will  probably 
result  in  lower  final  costs  but  higher  average  cost  growth. 

The  cost  estimation  dilemma  is  simply  this:  Underestimation  of  target  costs  provides 
(generally)  more  accurate  estimates  of  costs  and  a lower  base  from  which  to  operate,  but 
higher  average  cost  growth  measurements.  On  the  other  hand,  estimates  which  are  accurate 
on  the  average  have  the  disadvantage  of  the  higher  base  but  the  advantage  of  lower  average 
cost  growth. 
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CHAPTER  IV 


THE  EFFECT  OF  THE  SHARE  RATIO  CONFIGURATION  ON  COST  GROWTH 

It  was  pointed  out  in  Chapter  I (Section  B)  that  the  S/R  and  RIE  are  determined  by 
negotiation,  so  that  it  is  reasonable  to  suppose  that  these  two  important  elements  of  the 
CPIF  type  contract  structure  reflect  the  degree  of  uncertainty  and  risk  inherent  in  the  work 
statement  of  the  contract  Unfortunately,  the  S/R  and  the  RIE  are  not  independent,  and  in 
fact  the  lower  end  point  of  die  RIE  depends  quite  strongly  on  the  S/R  since,  by  law,  the 
profit  may  be  no  more  than  10  percent  of  target  cost  (or  15  percent  for  R&D  contracts; 
ASPR  3405.6).  The  fact  that  the  minimum  target  and  maximum  profit  figures  as  a percent 
of  target  cost  are  almost  always  6 percent,  8 percent  and  10  percent  or  7 percent,  8'A 
percent  and  10  percent,  respectively,  lends  reasonable  doubt  to  the  assumption  that  these 
figures  are  arrived  at  by  a strict  negotiation  process.  Therefore,  given  a percent  minimum, 
target  and  maximum  profit  the  RIE  is  determined  by  the  S/R. 

In  other  words,  the  S/R  and  the  RIE  are  too  closely  related  to  be  treated  as 
independent  variables  in  a statistical  analysis,  so  that  this  chapter  is  devoted  to  an 
investigation  into  the  relationship  between  the  share  ratio  and  the  elements  of  cost  growth. 
An  indirect  analysis  of  the  RIE  will  be  described  in  the  next  chapter. 


Let 


T 


Contract  initial  cost 
Contract  adjusted  cost 
Contract  final  cost 

Percent  contractual  adjustments  = (C^  - Cj)  /Cj 
Contractor’s  percent  share  of  O/R 
Difference  between  contractor’s  share  of 
O/R  and  contractor's  share  of  U/R. 


The  meaning  of  the  variable  \£  is  explained  with  the  following  example.  Suppose  that  the 
share  ratio  for  a contract  is  70/30  for  overruns  and  80/20  for  underrun.  Then  X|  = 30,  and 
X2  = 30-20  = 10.  In  order  to  avoid  ambiguity,  if  a contract  had  broken  share  ratio  either  for 
overruns  or  for  underruns,  then  that  contract  was  not  used  in  this  analysis. 

The  analysis  involves  the  linear  regression  model,  Y = M + AX  j + BX2  + e,  where  the 
error  term  e is  assumed  to  have  zero  expectation  and  also  assumed  to  be  independent 
(between  contracts). 

Table  II  gives  the  results  of  this  regression  analysis.  The  analysis  indicates  a strong 
dependence  of  contractual  adjustments  on  the  variable  X£,  but  none  on  Xj;  i.e.,  the 
magnitude  of  the  S/R  for  overruns  does  not  affect  the  amount  of  contractual  adjustments 
but  ths  “degree"  to  which  the  S/R  is  broken  has  a pronounced  effect.  (The  U/R  S/R  is 
obviously  higldy  correlated  with  the  O/R  S/R.  Thus,  the  result  should  hold  for  U/R  as  well. 
This  was,  in  fact,  verified.) 
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TABLE  II 


Multiple  Regression  Analysis  of  Percent  Adjustments  (Less  Quantity  Changes)  w.r.t. 

Share  Ratio  Information 


Sample  sire  = 46 

•••  •• 

Y = 33.1 + .50  Xj-4.7  X2 

Standard  error  = 65.72 
S.D.  of  Xj  = 1.30 
S.D.  of  X2  = 2.70 

tj  = .39 12  = 1.75** 

*43, .05  = 168 

NOTE:  In  this  report,  one,  two,  or  three  asterisks  will  be  used  to  denote  estimated 

parameters  which  have  been  found  to  be  statistically  significant  at  the  .1,  .05,  or  .01  levels, 
respectively. 

The  result  in  Table  II  is  surprising  and  suggests  immediately  that  the  same  analysis  be 
conducted  with  O/R  as  the  independent  variable.  It  was  decided  to  conduct  a correlation 
analysis  between  O/R  and  adjustments  as  well.  The  O/R  regression  analysis  is  given  in  Table 
HI  and  the  correlation  analysis  in  Table  IV. 


Xj  = 16.66 

X2  = 1.135 


•See  Appendix  H,  cogt  growth  report  II. 

Before  elaborating  on  these  results,  it  must  be  pointed  out  that  this  sample  includes 
CPIF  contracts  with  multiple  incentives,  Le.,  schedule  and  performance  incentives  (as  well  as 
cost  incentives),  which  involve  the  trade-off  of  dollars  on  the  one  hand,  and  time  and 
performance  characteristics  (such  as  uir  speed  and  weight)  on  the  other  hand.  It  is 
impossible  to  include  these  incentives  in  the  present  model  because  of  the  lack  of  data. 

It  was  decided,  therefore,  to  conduct  a regression  analysis  on  only  those  contracts  in 
the  sample  with  incentives  relating  to  cost  but  not  relating  to  schedule  or  performance,  or 
on  those  contracts  which  did  have  multiple  incentives  and  for  which  the  exact  amount  of 
the  profit,  adjustments  and  overrun  due  to  the  cost  portion  of  the  incentive  was  given. 
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There  are  only  29  K’*  in  the  sample  which  satisfy  one  or  the  other  of  these  restrictions, 
and  of  these  29,  only  six  had  a "broken”  S/R.  Since  “high  risk”  contracts  tend  to  have 
multiple  incentives,  it  is  not  unHkrfy  that  this  subsample  is  the  “low  risk”  part  of  the 
original  sample.  Sine?  contractor's  behavior  is  highly  dependent  on  die  amoimt  of  his  risk, 
and  the  contractual  limitation  on  his  behavior  is  dependent  on  the  inherent  risk  involved,  a 
risk  factor  or  measurement  must  be  introduced  to  the  regression  model  in  order  to  retain  a 
reasonable  degree  of  reliability  on  the  inferences.  No  such  risk  measurement  is  available  at 
this  time.  Hence,  the  value  of  this  secondary  analysis  as  a means  of  improving  the 
information  derived  from  the  original  regression  analysis  is  open  to  question. 

The  result  of  the  analysis  involving  only  die  "cost"  contracts  is  given  in  Tables  V and 
VI.  None  of  die  regression  coefficients  is  statistically  significant  even  at  the  .1  level  That  is, 
with  this  data,  no  relationship  between  0/R  or  adjustments  and  S/R  or  RIE  is  detectable 
even  at  the  10  percent  level  of  significance. 

A schematic  illustration  of  die  first  regression  analysis  is  given  in  Figure  9.  The  sketch 
labeled  (b)  in  figure  9 represents  an  unbroken  share  ratio.  As  the  S/R  becomes  more 
"concave”  in  the  upward  direction,  as  depicted  in  (a),  the  tendency  toward  more 
contractual  adjustments  increases.  As  S/R  assumes  a profile  which  is  “concave"  in  the 
downward  direction,  (c)  there  corresponds  s decrease  in  the  amount  of  contractual 
adjustments. 

Sketch  (e)  is  again  supposed  to  represent  the  average  S/R  profile.  As  the  dope  in  the 
share  ratio  decreases,  (corresponding  to  a decrease  in  the  contractor’s  share  of  the  overrun 
and  underrun)  there  is  an  increased  tendency  to  overrun,  while  an  increase  in  this  slope  is 
accompanied  by  a decrease  in  overruns,  or  an  increase  in  underruns. 

The  profile  in  sketch  (d)  is  characteristic  of  contracts  of  higher  risk.  Therefore,  the 
indication  is  that  the  high  risk  contracts  produce  more  overrun  than  lower  risk  contracts. 

Sketches  (a)  and  (c)  characterize  contracts  with  low  target  cost  and  high  target  cost, 
respectively,  with  respect  to  the  expected  final  cost  [6;  62-64, 81-87].  Sketch  (a)  is  then 
representative  of  the  "buy-in"  situation.  One  would  expect  to  experience  more  than  the 
average  number  of  contractual  adjustments  in  this  situation. 
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TABLE  V 


t 

t 


Regression  Analysis  of  Percent  Adjustments  (Less  Quantity  Changes)  for  “Cost  Only” 
Incentives 

Sample  ske  = 29 

Y = 23.2 + .554  (X1)-.08(X2) 

S.  Err.  - 43.3 

S.D.  X1=.967 
S.D.  X2  = 2.19 

tj  = .572  t2  = -.04  F326  = 5.6**« 

*3, 26, .01  =4-64 
TABLE  VI 

Regression  Analysis  of  Percent  O/R  (Prorated)  for  “Cost  Only”  Incentives 
Sample  sixe  = 29 

Y = -6.8  + .34  Xj  • .017  X2 
Standard  Error  = 162.8 

S.D.  Xj  = 3.64 
S.D.  X2  = 8.12 

tj  = 1.19  t2  = .027  F = .32 

*26,.!  = 13 
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SCHEMATIC  ILLUSTRATION  OF  RELATIONSHIP  BETWEEN  DEGREE  OF  SHARE-RATIO 


BREAK  AND  BETWEEN  SHARE  RATIO  AND  OVERRUN 


MORE 

ADJUSTMENTS 


AVERAGE  LESS 

ADJUSTMENTS  ADJUSTMENTS 


TC  = Target  Cost;  TP  * Target  Profit 


FIGURE  9 
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As  an  aid  in  drawing  firm  conclusions  from  this  analysis,  it  would  be  helpful  to  know 
whether  the  broken  S/R  is  caused  by  a decrease  in  contractor's  S/R  for  O/R  or  an  increase 
in  his  S/R  for  U/R  or  from  both.  Ih  an  effort  to  determine  this,  the  average  S/R  for  the 
contracts  with  unbroken  S/R  was  compared,  with  those  contracts  that  had  a broken  share 
ratio  both  for  0/R  and  U/R,  by  means  of  a t-test  The  results,  given  in  Table  VII  indicate 
that  only  the  S/R  for  U/R,  in  the  “broken”  case,  is  significantly  different  from  the  average 
“unbroken”  S/R.  In  other  words,  the  broken  S/R  is  used  more  as  a device  for  increasing  the 
contractor's  reward  for  undem'n,  rather  than  increasing  his  penalty  for  any  overrun. 

Note,  that  the  standard  deviation  of  the  broken  S/R  for  0/R  is  much  lower  than  the 
standard  deviations  for  the  other  two  cases.  Even  though  this  difference  (cr  ratio)  of 
variances  was  not  found  to  be  significant  at  the  10  percent  level  with  the  usual  F-test,  it  did 
seem  advisable  to  calculate  and  display  the  Rehrens-Fischer  t-statistic  along  with  the  usual 
“pooled”  t-«tatistic.  Neither  of  these  t-values  approach  significance  at  any  “reasonable" 
level. 

TABLE  VII 


f Comparison  of  S/R’s  for  the  Broken  and  Unbroken  Cases. 


Type 

Ave. 

n 

Var. 

t (D.F.) 

Unbroken  S/R 

17.2 

37 

64.9 

_ 

Broken  S/R  (O/R) 

15.8 

9 

18.0 

1.1  (30)  Behrens-Fiscber 
.76  (44)  Pooled 

Broken  S/R  (U/R) 

22.8 

9 

59.6 

-1.87* **  (44) 

*44,  .05  = 168 

The  interpretation  of  the  collective  analysis  presented  in  this  chapter  seems  formidable. 
The  main  findings  are  as  follows: 

(1)  There  is  a significant  increase  in  contractual  adjustments  when  the  S/R  ior  U/R  is 
increased,  and  (2)  a significant  decrease  in  0/R  (or  increase  in  U/R)  corresponding  to  an 
increase  in  the  contractor's  S/R.  Finally,  (3)  it  was  found  that  contractual  adjustments  and 
0/R  are  statistically  independent  and  (4)  no  significant  effects  were  detected  on 
adjustments  and  O/R  due  to  the  S/R  when  only  the  cost  portion  of  the  O/R  and 
adjustments  are  considered. 


The  remit*  of  the  analysis  presented  in  this  chapter  are  not  unexpected,  and  the 
experienced  procurement  analyst  would  be  the  first  to  assert  that  This  chapter  offers  a 
statistical  confirmation  of  the  relationship  between  the  incentive  structure  and  risk  in  CPIF 
contracts.  ' 
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CHAPTER  V 


THE  RELATIONSHIP  OF  PROFIT  TO  SHARE  RATIO 
AND  RANGE  OF  INCENTIVE  EFFECTIVENESS 

In  the  introductory  statement  of  Chapter  IV,  it  was  explained  why  a direct  analysis  of 
the  RIE  is  considered  inappropriate.  The  indirect  analysis  in  this  chapter  will  first  of  all 
atte.npt  to  determine  how  the  final  costs  are  distributed  with  respect  to  the  RIE  and 
especially,  how  this  distribution  appears  with  respect  to  those  contracts  with  final  cost 
outside  the  RIE. 

There  is  one  other  indirect  analysis  suggested  by  this  approach  in  conjunction  with  the 
result  of  Chapter  II.  Since  the  correlation  between  negotiated  contractor’s  incentive  share 
and  that  computed  with  final  cost  figures  is  quite  low,  is  there  any  disparity  between  the 
contractor's  actual  profit  and  that  which  would  be  produced  by  the  share  line?  The  low 
correlation  mentioned  above  does  not  necessarily  imply  that  this  should  be  the  case  but,  in 
fact,  if  there  are  many  final  costs  in  the  sample  which  are  outside  the  RIE,  this  might  be 
expected. 

This  chapter  then  begins  with  an  examination  of  the  contractor’s  expected  profit  as 
related  to  the  RIE.  In  order  to  do  this,  it  was  decided  to  normalize  the  incentive  S/R  and 
RIE  so  that  the  TC,  TP,  Max  P,  Min  P,  and  upper  and  lower  ends  of  the  RIE  have  the  same 
value  for  each  contract.  This  was  done  for  all  contracts  which  had  no  break  in  the  S/R  either 
above  or  below  the  TC  and  for  which  the  complete  S/R  and  RIE  was  available  for  the  cost 
portion  of  the  contract  only.  This  “normalized"  S/R  line  is  shown  in  figure  10.  The  purpose 
of  this  data  transformation  is  so  that  a sample  of  more  than  one  observation  could  be 
subject  to  analysis. 


29 


NORMALIZED  SHARE  LINE  USED  IN  THE 
PIECE-WISE  LINEAR  REGRESSION;  PROFIT  VS.  COST 


Max  L 
Profit! 


Target! 

Profit 


\ 

\ 


Min 

Profit! 


X 


Lower  Target  Upper 

RIE  Cost  RIE 


4» 


FIGURE  10 


30 


Figure  1 1 shows  the  transformed  data  for  the  30  observations.  The  degree  of  scatter 
about  the  normalized  share  Line  appears  quite  high.  Several  reasons  for  this  are  aLluded  to  in 
Chapter  II  in  connection  with  the  correlation  analysis  of  negotiated  S/R  with  respect  to  the 
estimated  S/R.  The  one  which  would  appear  to  have  the  most  marked  effect  is  the  existence 
of  multiple  incentives;  he.,  incentives  which  reward  (or  penalize)  the  contractor  for 
completing  the  contract  ahead  of  (or  behind)  schedule  or  which  reward  (or  penalize)  the 
contractor  for  delivering  a product  which  performs  better  (or  worse)  than  required  by 
the  contract. 

Therefore,  the  contract  cost  figures  were  “cleaned  up"  so  that  they  would  reflect  only 
payments,  penalties,  profits,  and  costs  made  under  the  cost  portion  of  the  incentive.  Figure 
12  shows  this  “cost-portkm-only”  data.  There  appears  to  be  no  significant  change  in  the 
degree  of  scatter,  but  one  of  the  data  points  has  moved  from  outside  the  RIE  (on  the  high 
side)  to  within  the  RIE,  Thus  only  four  of  the  30  points  are  above  the  RIE;  and  two  lie 
below  the  RIE. 


It  would  be  desirable  at  this  point  to  test  the  hypothesis  that  this  data  could  have  been 
generated  by  the  normalized  cost  share  line,  and  also  to  test  similar  hypotheses  for  the  two 
cases,  within  and  outside  the  RIE,  separately.  Gince  there  are  only  two  points  below  and 
four  points  above  the  RIE,  and  these  points  are  generated  by  a “different”  straight  line 
model  than  are  those  within  the  RIE,  a statistical  method  which  analyzes  several  linear 
models  in  combined  form  as  a “piecewise-linear”  model  is  required. 

A search  for  such  a model  was  conducted  in  all  of  the  available  literature,  but  none  was 
found.  Therefore,  this  statistical  technique  was  developed  within  APRO.  (See  Ap|iendix  HI 
for  details  or  reference  [9]  for  a complete  description.) 

The  Piecewise  Linear  l^odel  (PLM)  employed  in  this  analysis  is  as  follows, 


r 

aj  + bjX  1 £ x ■£  2 

a2  + ^2^  > X ^ 2 , subject  to  the  constraint, 

V 


aj  + 2b  ^ = a2  + 2b2,  where  y represents  final  normalized  profit  and  X represents  find 
normalized  coct. 

It  should  be  pointed  out  that  the  two  data  points  to  the  left  of  the  y-axis  result  from 
underruns  which  brought  the  final  cost  below  the  lower  limit  of  the  incentive  range,  and 
hence  the  contractor  received  the  maximum  profit.  Whether  all  points  in  this  region  would 
be  exactly  on  the  theoretical  profit  line  or  not,  cannot  be  ascertained.  Therefore,  only  the 
28  points  which  correspond  to  final  costs  within  the  RIE  or  above  the  upper  limit  of  the 
RIE  were  subjected  to  analysis. 
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The  normalized  share  line  is  depicted  in  figure  12  by  the  solid  line,  and  the  estimated 
“expected  share  line”  is  represented  by  the  dashed  line.  The  regression  analysis  (table  VM) 
indicates  a significant  difference  between  the  parameter  of  the  solid  and  the  dashed  line  at 
fire  5 percent  lend. 

It  is  important  to  know  which  of  the  parameters  contributes  to  the  significance 
indicated  by  the  large  F-value.  It  is  equally  important  in  this  particular  analysis,  to  know 
which,  if  any,  of  the  parameters  is  not  significant  To  accomplish  this,  the  parameters  were 
tested  individually.  The  results  are  given  in  Table  IX. 

For  future  reference,  it  is  mentioned  that  the  average  negotiated  S/R  for  the  contracts 
within  and  above  the  RIE  are  17.5  percent  and  10  percent,  respectively.  The  percent 
increase  in  target  cost  due  to  contract  modifications,  less  O/R  and  quantity  changes  are  25.5 
percent  and  2.3  percent  for  contracts  with  final  costs  within  and  above  the  RIE, 
respectively.  Finally,  the  percent  O/R  was  1.4  percent  and  45  percent  respectively,  for  those 
contracts  with  final  costs  within  and  above  the  RIF,. 
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The  DATA  WITH  NO  QUANTITY  CORRECTION 


Normalized  Profit 


V 


TArtL*-.  vm 


Piecewise  Linear  Regression  Analysis  of  Normalized  Cost  and  Profit 
Test  on  Entire  ModeL 


y 


.0185  + 

.7004  (x-  .9292) 

.1819  - 

.157  (x-  Z.b5) 

,x  > 

n = 28 

m^  = 24 

m<>  = 1 

2 

2 


Standard  Error  = .237 


Ho:  bj  = -1.  a2  = 0^2  = o. 


F = 3.16#* 


^3, 25, .05 


2.99 


The  F-ratio  is  significant  at  the  5 (>ercen.v  level.  In  an  effort  to  determine  which  of  the 
parameters  contribute  the  mo»t  to  li.is  significance,  the  parameters  were  tested  individually. 
The  results  -c  g ' en  in  Tame  IX. 
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TABLE  IX 


Piecewise  Linear  Regression  and  Hypothesis  Test  on  Individual  Pieces  of  the  Model,  With 
and  Without  Constraints. 


n = 28,  nij  = 24,  = 4. 


Ho;  b!  = -l. 

F = 6.9** 

Ho;  aj  = 1 
F = 3.646** 
Ho;  *2  - o 

b2  = ° 
F = 3.15 
Ho:  2 = ° 

F = .139 
Ho;  1>2  = o 
F = .302 


(with  constraint) 

F 05,1,25  =4-24 

(no  constraint,  corrected  for  mean) 

F.05,2,22  = 3‘44 

(with  constraint) 

F.  05,2,25  = 3-39 

(with  constraint) 

(with  constraint) 


It  is  clear  that  the  slope  estimate  obtaineu  from  the  points  corresponding  to  final  costs 
which  are  within  the  RIE  is  causing  the  significance.  The  indication  is  that  the  share  of  both 
O/R  and  U/R  which  the  contractor  is  acluai.)  receiving  is  less  than  the  negotiated  share 
ratio.  This  does  NOT  mean  that  the  Government  changes  or  disregards  the  terms  of  the 
contract.  It  DOES  mean  that  perturoutlo.^  ..re  hro^hi  about  which  are  not  (completely) 
governed  by  the  contractual  agreciwciu  out  wiucn  do  ct.ange  either  the  cost  or  profit  or 
hoih.  One  example  of  this  ty^c  of  ciImi0c  ..•»  ».»<*  contract  renegotiation.  (See  also  Chapter  I, 
Section  C.) 
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CHAFFER  VI 

CONCLUSIONS  AND  RECOMMENDATIONS 

The  analysis  described  in  Chapters  II  through  V was  performed  to  answer  the  questions 
posed  in  Chapter  L Section  C.  Although  several  of  the  questions  are  answered  very  directly, 
several  of  them  wiil  require  additional  explanation.  The  major  findings  will  first  be 
summarized  so  that  they  may  be  easily  referred  to  during  a reading  of  the  immediately 
following  section. 

A.  Major  Findings. 

1.  The  estimated  correlation  coefficient  between  negotiated  share  ratio  and  the  share  ratio 
estimated  from  the  final  cost  information  from  CPIF  contracts  is  .15  with  a t value  of  .79. 
Even  when  the  data  is  restricted  to  those  contracts  which  experienced  an  overrun  and  which 
had  an  “unbroken"  negotiated  share  ratio,  the  estimated  correlation  coefficient  is  less  than 
.25  with  a t-value  of  about  .75. 

2.  When  the  cost  growth  average  and  mode  are  both  computed  as  a percent  of  initial  target 
cost,  the  average  was  found  to  be  20  percent  greater  than  the  mode.  Since  estimates  of 
“most  probable  cost"  (i.e.,  the  mode)  are  used  as  contract  target  costs,  and  since  we 
measure  “average”  (i.e.,  mean)  cost  growth,  there  appears  to  be  a built-in  cost  growth  base 
of  20  percent  on  all  CPIF  contracts. 

3.  The  data  yielded  a significant  positive  correlation  between  the  contractor’s  share  of 
underrun  and  contractual  adjustments. 

4.  The  data  indicated  a statistically  significant,  negative  correlation  between  O/R  and  the 
contractor’s  chare  fer  ovt.-runs 

5.  Contractual  adjustments  and  overruns  are  statistically  independent. 

6.  The  share  of  both  the  underrun  and  the  overrun  which  the  contractor  actually  receives 
is,  on  the  average,  less  than  the  negotiated  share,  within  the  range  of  incentive  effectiveness. 

B.  Discussion. 

Some  of  the  questions  posed  in  Chapter  I,  Section  C,  involve  the  intent  or  desires  of 
contractors.  Since  intents  and  desires  are  not  measurable,  the  questions  which  involve  them 
are  not  amenable  to  direct  statistical  analysis.  For  this  reason,  and  because  questions  of  this 
type  are  somewhat  accusatory  toward  the  contractor,  it  would  seem  that  the  only  way  to 
answer  the  questions  with  a high  degree  of  confidence  would  be  in  a court  of  law.  This  is 
clearly  impossible  within  the  confines  of  this  project,  so  a search  for  indicators  of  an 
indirect  sort  has  been  employed  in  this  study. 

It  is  natural  to  assume  that  a contractor  ceases  to  attempt  to  control  costs  when  he 
loses  the  incentive  to  do  so;  i.e.,  when  the  contract  cost  has  moved  to  a point  outside  the 
range  of  incentive  effectiveness  (Itl£).  There  is  an  indication  of  this  in  the  analysis. 
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The  data  in  figure  12  indicate  that  the  spread  of  final  coats  which  were  above  the  RIE 
is  large.  Furthermore,  the  negotiated  S/R  for  these  contracts  was  only  10  percent  as 
opposed  to  the  17.5  percent  for  the  others. 

A positive  identification  of  the  buy-in  also  proved  elusive.  The  phrase  “buy-in”  refers 
to  the  situation  that  occurs  when  a contractor  bids  so  low  on  a particular  proposal  that  he  is 
virtually  insured  of  being  awarded  the  contract,  thereby  literally  “buying”  the  contract 
(away  from  competition). 

One  indication  of  the  buy-in  is  an  S/R  configuration  similar  to  that  shown  in  figure  9 
(a);  that  is,  an  S/R  which  is  concave  upwards  and  a target  cost  which  is  nearer  to  the  lower 
end  of  the  RIE  than  to  the  upper  end.  The  regression  analysis  in  Tabic  V shows  that  as  the 
S/R  becomes  more  concave  upward,  the  number  of  contract  adjustments  increases.  Table 
VII  indicates  that  the  concavity  is  most  affected  by  die  U/R  portion  of  the  S/R,  and  since 
underrun  increases  with  the  S/R,  the  existence  of  the  “buy-in”  pattern  seems  to  be  verified. 

The  analysis  of  Chapter  III  indicates  that  the  use  of  estimates  of  the  most  probable 
contract  cost  for  use  as  the  target  cost,  as  opposed  to  using  estimates  of  the  expected  cost, 
may  produce  as  much  as  21  percent  cost  growth. 

The  regression  analysis  reported  in  Chapter  IV  did  not  provide  evidence  of  a 
relationship  between  contractual  adjustments  and  the  contractors  negotiated  share  of  the 
overrun,  although,  the  O/R  was  found  to  decrease  with  an  increase  in  the  contractor’s 
negotiated  share.  It  is  felt  that  this  is  produced  by  the  “buy-in”  phenomena  described  above 
rather  than  uncertainty. 

A relationship  was  found  to  exist  between  the  degree  of  break  in  the  share  ratio  and 
the  amount  spent  on  contract  modifications.  When  the  break  in  the  share  line  produced  a 
“concave  upward”  configuration  (see  figure  9,  Chapter  IV)  the  amount  spent  on 
modifications  was  highest  But  as  the  share  line  assumed  the  form  of  a straight  line  and 
passed  on  to  the  “concave  downward”  configuration  the  modifications  steadily  decreased 
on  the  average.  If  the  overrun  followed  an  identical  pattern,  then  this  effect  could 
reasonably  be  attributed  to  “uncertainty,"  while  if  O/R  were  low  (i.e.,  high  U/R)  whenever 
the  share  line  configuration  was  concave  upward,  then  the  aforementioned  effect  could  be 
assumed  to  be  due  to  the  “buy-in"  or  “low  initial  cost-high  modifications"  type  of 
contractor  behavior.  No  dependence  of  O/R  on  either  the  share  line  configuration  or  on 
contract  modifications  was  detected.  Since  both  of  these  should  produce  the  effect  on 
contract  modifications  noted  above,  while  their  effects  on  O/R  are  counteractive,  the  overall 
result  is  a general  confirmation  of  the  possible  existence  of  both  causes— uncertainty  and 
buy-in. 

The  “piecewise"  linear  regression  which  is  described  in  Chapter  V,  was  conducted  in  an 
effort  to  determine  whether  or  not  average  profits  for  contracts  with  final  costs  above  the 
RIE  are  high  enough  to  justify  a recommendation  to  extend  the  upper  end  of  the  RIE.  The 
analysis,  however,  indicated  that  the  average  profit  in  this  final  cost  range  did  not  differ 
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significantly  from  that  which  was  negotiated,  h was  also  discovered  that  the  average  profit 
for  contracts  experiencing  0/R  and  U/R  were,  respectively,  greater  than  and  less  than  the 
profit  dictated  by  the  negotiated  share  line  within  the  RIE.  It  is  pointed  out  that  this  effect 
could  not  be  entirely  due  to  noise  in  the  data  but  must  be  at  least  partially  generated  by  a 
systematic  mechanism. 

In  any  case,  there  is  no  apparent  reason  for  recommending  the  extension  of  the  RIE. 

Chapter  II  provides  a positive  basis  for  rejecting  the  use  of  estimated  share  ratio 
obtained  from  final  contract  cost  figure;  instead  of  the  negotiated  share  ratio,  when  the 
latter  is  not  available  to  die  analyst. 

C.  Recommendations. 

The  results  of  this  study  are  of  such  a nature,  that  clear-cut  policy  recommendations 
would  be  exceedingly  difficult  to  make.  Therefore,  the  recommendations  are  pointed  at 
either  senior  Army  policy  makers  for  consideration  in  policy  formulation,  or  else  for 
analysts  and  researchers  who  offer  advice  to  the  policy  makers  or  who  conduct 
investigations  in  logistics  problems. 

1.  It  is  recommended  that  senior  DA  procurement  analysts  be  made  aware  of  the 
relationship  between  the  mean  and  the  mode  in  CPIF  cost  data.  The  obvious  but  simplistic 
recommendation  that  “expected"  costs  should  be  used  instead  of  "most  probable"  costs 
will  be  avoided  here,  because  of  the  possible  effect  of  the  resulting  higher  target  costs  on  the 
final  costa.  Any  further  policy  recommendation  will  require  more  study  tempered  with 
sound  procurement  judgment 

2.  It  is  recommended  that  procurement  analysts  be  made  aware  of  the  uisparity  between 
the  contractors  negotiated  share  of  overrun  and  underrun  and  the  (smaller)  share  which  he 
receives  on  the  average.  A more  extensive  recommendation  will  not  be  given  since  this 
disparity  works  to  the  advantage  of  the  Government  in  the  underrun  situation.  There  is  also 
the  possibility  of  the  existence  of  hidden  "trade-offs"  between  the  Government  and  the 
contractor,  which  are  not  measurable  from  the  data,  but  which  work  to  the  benefit  of  the 
Government. 

3.  It  is  recommended  that  studies  performed  within  the  Department  of  the  Army 
involving  the  affect  of  the  share  ratio  in  CPIF  contracts  on  cost  growth  should  use  the 
negotiated  share  ratio  obtained  from  the  contract  files.  Analysis  which  is  based  on  share 
ratios  estimated  from  the  final  cost  data  may  be  highly  inaccurate. 
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This  appendix  it  a description  of  the  density  estimator  used  in  this  report  which  was 
taken  directly  from  the  paper  entitled  “On  Estimation  of  a Probability  Density  Function 
and  Mode"  by  Emanuel  Paraen,  which  appeared  in  the  Annals  of  Mathematical  Statistics, 
Volume  33,  1962,  [11].  Only  the  barest  details  are  given  here  for  brevity.  The  interested 
reader  is  referred  to  the  original  paper. 

Suppose  that  X is  a R.V.  with  density  function  f(x),  and  xj,  X2, . . .,  xn  is  a random 
sample  of  observations  of  X.  Paraen  proposes  a general  class  of  weighting  functions  K(y)  to 
estimate  f(x)  as  follows: 


Kit-*,) 
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where  fn(y)  is  the  estimate  of  f(y),  and  h = h(n)  is  a function  of  the  sample  size;  Paraen 
shows  that  f„(y)  is  asymptotically  unbiased  if  K(y)  is  bounded,  if  the  total  integral  of  K(y) 
converges  absolutely,  and  if  lim  n — }►  <x>  {{(>\)  ~ O , 

Uin  y co|  ~c  i J /<  - 1 , 

— co 

Further,  the  variance  may  be  estimated  from 


if  f(x)  is  continuous  at  x. 

Paraen  suggests  seven  specific  forms  for  K(y).  A number  of  Monte  Carlo  simulation 
runs  indicated  that  n = 12,  24,  and  36,  the  function  given  by 

_j_  i ^n-jA) \2 

zjt  \ / 

is  noticeably  superior  to  the  others.  This  one,  incidentally,  also  has  the  smallest  variance  as 
reported  in  Paraen’s  paper,  and  is  approximately 
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The  sample  mode,  6 n,  is  obtained  by  finding  fn(  6 n)  = max  (- oo<  x<  oc 
In  this  study,  a simple  grid  procedure  was  used  and  & n was  obtained  to  within  .0001 
with  three  iterations  of  21  estimates  of  fn(x). 

The  sample  mode  (as  are  the  values  of  fn(x))  are  asymptotically  unbiased  and 
asymptotically  normally  distributed.  The  variance  is  approximately: 

2 

Va  W0-*)  7^  ~r*r  ( t < »)/  { (eSj 

Since  the  estimate  involves  the  second  derivative  of  f(x),  and  the  distributions  involved  in 
this  study  were  rather  sharply  peaked,  estimating  the  second  derivative  is  difficult.  Attempts 
were  made  at  modifying  Parzen’s  estimator  by  estimating. 

(f(x+h)  - f(x))/h 


for  the  first  derivative  and  similarly  for  the  second  derivative.  These  estimates  were  rather 
unstable,  but  not  completely  useless. 

The  h value  used  in  this  study  is  taken  from  studies  performed  in  the  Statistics 
Department  at  VP1  & SU  at  Blacksburg,  Virginia  (cf.  [1]). 
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Appendix  II 


The  purpose  of  this  appendix  is  to  indicate  how  an  estimator  which  underestimates,  on 
the  average,  can  be  better  than  another  estimator  which  does  not  underestimate  or 
overestimate  on  the  average.  A specific  example  will  be  given. 

Let  X|,  X£,. . . Xn  represent  a random  sample  of  the  r.v.  X with  unknown  parameter 
0,  and  frequency  function  f(x;  0).  Further,  let  0 = g(xj,  . .,  x^  represent  an  estimator  of  0. 
ft  it  said  to  be  an  unbiased  estimate  of  0 if  its  expected  value  is  0t  Le.,  E(ft)  = 0.  The 
quantity  E (ft-0)2  is  called  the  “mean  squared  error,9'  MSE,  and  if  $ is  unbiased  for  0,  this  is 
also  the  variance  of  0,  or  Var  (0). 

Hie  problem  of  comparing  the  quality  of  different  estimators  is  bothersome,  because 
there  is  no  objective  universally  accepted  measure  of  quality.  When  comparing  two  unbiased 
estimators,  the  better  one  is  conceded  to  be  the  one  with  the  smaller  variance,  because  the 
average  squared  deviation  of  this  estimator  is  smaller.  In  other  words,  it  will  be  “closer  more 
often."  The  logical  extension  of  this  idea  is  to  simply  compare  the  average  squared  deviation 
(MSE)  disregarding  the  bias  altogether.  As  a matter  of  fact,  it  can  happen  that  the  MSE  of 
an  unbiased  estimator  can  be  reduced  by  introducing  a bias.  This  can  be  seen  in  the 
following  development. 

The  MSE  of  an  estimator,  0,  can  be  written, 

MSE(ft)  = E[ft-E(ft)]2  + fE(ft)-0]2  (i) 

Now  multiply  the  estimator  § by  the  (as  yet  unknown)  constant,  A,  to  form  the  new 
estimator,  ft  = A§.  Let  ra  = MSE  (0).  One  method  of  finding  thfct  value  of  A which  will 
minimize  m (if  there  is  such  a value),  is  to  “set”  the  first  derivative  of  m with  respect  to  A 
equal  to  zero,  and  solve  for  A.  Thus, 
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One  example  of  an  estimator  winch  satisfies  this  last  equation  is  the  variance  estimate 
for  a normal  r.v. 

Let  X be  normally  distributed  with  mean  jl  and  variance  (both  unknown).  An 
unibased  estimate  of  O'  ^ is 
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Since  (n-1)  sfycr  2 has  the  chi-square  distribution  with  (n-1)  degrees  of  freedom,  then 
rire  variance  of  is  2cr  ^/( n-1)  and  thus  the  second  moment  about  the  origin  (from  (1)),  is 
cr  ^(n+l)/(n-l).  Substituting  these  values  into  (2)  yields: 


C r‘f  (Tt  + i)/(n-<) 
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h 1-1 


Therefore,  the  MSE  of  S^  as  an  estimator  of  o'  ^ may  be  reduced  by  multiplying  it  by 
the  number  (n-l)/n+l).  In  other  words, 
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has  smaller  MSE  than  S^,  and  S'' ^ underestimates  O'  because  S^  is  unbiased  and, 
therefore, 


r-  / 'A  _ Jil  1 
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< cr 


Finally,  MSE  (S/^)=  2 cr  ^/(n+1)  which  is  less  than  MSE  (S^)  = 2 o'  ^/(n-1). 
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This  appendix  is  a description  of  the  piecewise  linear  model  used  in  die  analysis  of 
Chapter  IV.  Full  details  may  be  obtained  in  Reference  9.  The  model  is  of  the  form 

r«l  + *>1  (x  • »l).  X * X* 

y ={ 

1*2  + b2  <*  * *2>*  * > ** 
subject  to  the  constraint 


*1  + bj  (x*  - %)  = a2  + b2  (x*  - x2). 

The  number  x*  is  assumed  known. 

It  is  convenient  to  employ  metric  notation.  If  the  x-matrix  is  written: 
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and = [aj,  bj,  a2,  b2],  then  the  model  may  be  written: 


y.  = x J , subject  to  T/X  = o,  where  T-[l,x#-xj, -1,  -x*  + x2j  and 
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Notice  that  for  i 5 m,i|<  x*  and  for  i > m,  *|  > x#. 

The  method  of  LaGrangian  multiptiera  appliea  to  this  minimiution  problem  subject  to 
tiie  constrar  .1  ft  is  necessary  then,  to  And  the  extreme  value  of 

(y.-x,e)'(y .-*.■*)  + at, ± , 

where  A is  the  LaGrangian  Multiplier.  Differentiation  with  re*j>ect  toy  J and  \ yields 
the  “constrained"  normal  equations: 

X ■f'  ‘/z  i A = A ^ 

T/l  - £ 

Solving  for  (h.  one  obtains 

-3  •-  (Xx)  x'tj  - V/  (Xxl  T ' ? 

Vx  [T(A'X)-'T'J-'[1  (X'xr'x'd . 
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If  - (y  X)  X the  usual  least  squares  estimate  of^$  , then 

/!  --  /l  -(x'x)'‘r  '[r  (x'x)" '•/•"]''  r /l . 
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Obtaining  the  distributional  properties  of  ,'i.  is  straightforward  but  cumbersome  (see 
reference.).  It  may  be  shown  that  the  error  sum  of  squares  subject  to  the  constraint  is: 
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For  testing  the  hypothesis  Ho:  7©  jk  Ac  Subject  to  ! ■ ' i Let 
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Then  the  teat  rUtistic  is 
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01  ~ ■ l , where  nrv  = Rank  of  T_.  The  teat  statistic 

m. 


is  distributed  as  F with  n - p+1  and  m degrees  of  freedom  and  the  noncentrality  paramete 

is: 
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